Smoothing methods for a morpho-statistical approach of automatic diacritization Arabic texts (Méthodes de lissage d'une approche morpho-statistique pour la voyellation automatique des textes arabes) [in French]
نویسندگان
چکیده
We present in this work a new approach for the Automatic diacritization for Arabic texts using three stages. During the first phase, we integrated a lexical database containing the most frequent words of Arabic with morphological analysis by Alkhalil Morpho Sys which provided possible diacritization for each word. The objective of the second module is to eliminate the ambiguity using a statistical approach in which the learning phase was performed on a corpus composed by several Arabic books. This approach uses the hidden Markov models (HMM) with Arabic unvowelized words taken as observed states and vowelized words are considered as hidden states. The system uses smoothing techniques to circumvent the problem of unseen word transitions in the corpus and the Viterbi algorithm to select the optimal solution. The third step uses a HMM model based on the characters to deal with the case of unanalyzed words. Mots-clés : Langue arabe, voyellation automatique, analyse morphologique, modèle de Markov caché, corpus, lissage, algorithme de Viterbi.
منابع مشابه
Evaluation of a possibilistic classification approach for Arabic texts disambiguation (Evaluation d'une approche de classification possibiliste pour la désambiguïsation des textes arabes) [in French]
Morphological disambiguation of Arabic words consists in identifying their appropriate morphological analysis. In this paper, we present three models of morphological disambiguation of non-vocalized Arabic texts based on possibilistic classification. This approach deals with imprecise training and testing datasets, as we learn from untagged texts. We experiment our approach on two corpora i.e. ...
متن کاملL'alignement des documents médiévaux
RÉSUMÉ. Le but de l’alignement des textes est la mise en correspondance des sous-parties similaires de deux ou plusieurs traductions ou versions d’un même écrit. La plupart des méthodes utilisées dans la technique d’alignement reposent sur l’analyse statistique des fréquences de mots ou de caractères, ou sur la cooccurrence des chaînes que ceux-ci constituent. Afin d’en améliorer l’efficacité, ...
متن کاملExploitation de l'échelle d'écriture pour améliorer la reconnaissance automatique des textes manuscrits arabe
RÉSUMÉ. Les documents manuscrits arabes présentent des défis spécifiques pour la reconnaissance du fait de la nature de l'écriture cursive et d'autres facteurs, comme la taille de l'écriture. Une des plus grandes bases étiquetées des documents manuscrits arabes, la base de données NISTOpenHaRT inclut de grandes variabilités dans la taille du texte inter et intra mots et lignes. Nous proposons ...
متن کاملA Mixed Morpho-Syntactic and Statistical Approach to Chinese Named Entity Recognition (Une approche mixte morpho-syntaxique et statistique pour la reconnaissance d'entités nommées en langue chinoise) [in French]
متن کامل
Identification of Arabic/French Handwritten/Printed Words using GMM-Based System
The discrimination between languages is one of the first steps in the problem of automatic documents text recognition. In many documents, such as bank checks and application forms, printed and handwritten texts are mixed. In this paper, an automatic identification system of Arabic and French words in both handwritten and printed script based on Gaussian Mixture Models (GMMs) was presented. A fi...
متن کامل